Stability of Group Representations and Haar Spectrum

If U and V are commuting unitary representations of locally compact abelian groups S and T, new representations of S (perturbations of U) can be obtained from composition with images of U in V. If most of these representations are equivalent to U, U is said to be V stable. We investigate conditions which, together with stability, ensure that U has (uniform) Haar spectrum. The principal applications are to dynamical systems which possess auxiliary groups with respect to which motion is stable. 0. Introduction. The notion of a dynamical system, which is stable with respect to the action of a compact abelian group, was introduced and exploited in [l] to show that under certain conditions the associated group representation has countable Lebesgue spectrum in the orthocomplement of the space of functions invariant under the compact group. In particular this method led to the conclusion that ergodic unipotent affine transformations on nilmanifolds have countable Lebesgue spectrum in the orthocomplement of the eigenfunctions, with a similar conclusion for nilflows; this latter result was first established by Green [2], The purpose of this paper is to investigate stability in the context of unitary representations of locally compact abelian groups. Our aim is to establish conditions which will ensure that a representation has a Haar spectrum with uniform multiplicity. Apart from the result mentioned above applications of our main theorems yield the well-known results: [3]j [4]. // U, V satisfy the Weyl commutation relation then U and V have Haar spectrum with uniform multiplicity. [5], [2]. The horocycle flow has Lebesgue spectrum with uniform multiplicity in the orthocomplement of the constant functions, (Actually, the multiplicity is infinite but our methods will not yield this.) Let S, T be two locally compact separable abelian groups, and let X, V be their duals. Let U, V be unitary representations of S, T, in the Hubert space H, which commute with one another (i.e., U V = VU for all s e S, t € T or, alternatively, U and V ace restrictions to 5" and T of a representation of S x T). In Received by the editors August 30, 1971. AMS 1970 subject classifications. Primary 22D10, 28A65.